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Writing (Grades 3, 6, 9 and'12). Student performances 
were judged by the markers to be weak in all four grades 
on the short answer section of ‘the writing test, which 
assessed abilities in capitalization, spelling, punctuation 
and sentence structure. Performances were stronger on the 
longer writing assignment, which asked students to write a 
story in grade 3, a story or‘description in grade 6, and an 
exposition in grade 9, and an argument in grade 12. All 
students achieved high scores on the writing assignment for 
limiting content to the topic and writing legibly. Alberta 
students appeared to be more successful at using their own 
language in writing than at finding errors in or manipulat- 
ing a structured assignment. The performance on the "Writ- 
ing assignment" part. of ‘the test was generally satisfactory 
but there was some concern about the fact that only 74 per- 
cent of grade 12 students performed at the functional level 
or better. (Students whose writing was rated as "functional" 
were able to communicate at a basic level but needed in- 
struction in some aspects of writing skills.) 


ABSTRACT 


At the time this study was being conducted, the 
Alberta Education curriculum guides for language arts 
stated the program objectives in rather general terms. The 
researcher, therefore, began by compiling a comprehensive 
list of specific objectives suitable for use in developing 
tests of reading and writing at the grades 3, 6, 9, and 12 
levels. The steering committee for the study, with the 
assistance of the Curriculum Branch of Alberta Education 
and its language arts coordinating committee, then ranked 
the measurable objectives in terms of their importance to 
the total program. When the committees had established 
priorities, the researcher had available a selected list of 
objectives for which appropriate test items could be devel- 
oped. 


The researcher used standard piloting and field test- 
ing procedures to investigate administrative and scoring 
techniques and to determine the need for revisions. The 
Manual scoring technique required for the writing tests 
presented a special challenge. To ensure fair and objec- 
tive marks, two specially trained markers scored each com- 
position using two different marking scales. In the event 
of a major discrepancy between the two marks, a third 
marker scored the composition. The researcher considers 
both the reading and writing tests to be superior to those 
available commercially, because of the greater face-validity 
for the Alberta curriculum and because of the unique marking 
techniques which were employed. 


The reading and writing tests were administered to each 
of grades 3, 6, 9 and 12 in a random sample of 284 schools 
on May 17, 1978. 


Reading (Grades 3, 6, 9 and 12). Student performance 
on the reading items were judged by the markers to be 
generally satisfactory at all four grade levels tested, and 
the performance in phonics at the grade 3 level was particu- 
larly satisfactory. Alberta students appeared to be improv- 
ing, or at least maintaining, their reading skills as they 
progressed through the suffixes to obtain word meanings in 
grade 6; in identifying directly stated details on maps and 
charts in grade 9; and in identifying implied elements of a 
passage in each of grades 3, 6 and 9. 


PREFACE 


The Minister's Advisory Committee on Student Achievement 
(MACOSA) was established by ministerial order in October 1976 in 
response to growing concerns expressed by the public at large, 
government, labor, business, students and educators regarding the 
quality and standards of basic education in Alberta. 


MACOSA commissioned a number of studies, primarily to 
provide basic information for a summary of current levels of 
achievement in Alberta and to provide baseline data for future 
assessment. These studies fell into three categories: (1) 
preliminary studies, (2) achievement studies, and (3) other studies. 


The purpose of this study, entitled the Alberta Language 
Arts (Reading and Writing) Achievement Study, was to provide informa- 
tion about current levels of student achievement in reading and writ- 


ing in Alberta schools and to provide a data base for future assess- 
ments. 


This report, which represents the findings and conclusions 
of the researchers, was presented to MACOSA as information. 
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PART I 
EXECUTIVE SUMMARY 


I. INTRODUCTION 


I.l.1 Background | 

The Minister's Advisory Committee on Student Achievement 
(MACOSA) was established by Ministerial Order in Alberta in 
October 1976.° MACOSA was established in response to growing 
concerns on the part of the public, the Government, labor, business, 
students, and professional educators regarding education. Interest 
groups have been concerned that the standards for student achieve- 
ment may have declined in recent years, and that these standards 
may be inadequate for current and future needs of students and 
various educational and employment sectors. Interest groups have 
also been concerned that the standards are not clearly demonstrable 
given the varieties of student programs, marking procedures and 
sede, and that the means -oe reporting siident achievement may 
ea Pasccanate coe purposes Sick ae certification of broad and 
specific dienennas sn various curricular areas, career counselling, 
and selection procedures used by post-secondary institutions and 
employers. 

MACOSA commissioned several studies to facilitate the 
development of a summary of current levels of student progress in 


Alberta and to provide baseline data for future assessments. More 


specifically, on August 18, 1977, Alberta Education contracted the 
Canadian Institute for Research (CIR). to assess two selected areas 
of communication/language arts: reading and writing. 

CIR worked under the supervision of a MACOSA-appointed 
Steering Committee to gather available data needed to assess the 
current levels of reading and writing in grades 3, 6, 9 and 12 in 
Alberta, to interpret the data in terms of student achievement, 
and to compile the information together with recommendations into 


a report submitted to Alberta Education on September 30, 1978. 


I.1.2. Purposes 


Three major purposes paved cas guidelines for the general 
features of fic study's design. 
- To collect and nepore intovmation on the current levels 
of student achievement of selected obisetives an reading 


and writing in grades 3, 6, 9 and 12 in Alberta schools. 


- To produce useful and timely summaries of the current 
status of student achievement so that MACOSA has a basis 
_ for submitting reports and recommendations to the Alberta 


Minister of Education. 


- To construct a data base of student achievement informa- 
tion. which can be used as baseline information for future 


assessments. 
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I.1.3 Limitations of the Assessment 

The subjects involved in this study were limited to 10 
percent of the students at each of grades 3, 6, 9 and 12 in the 
province of Alberta. Therefore, the sample used in this study 
is yeRrescheaeiwe of the total student population within the 
limitations imposed upon it by the sampling procedures. 

The items on the instruments used in this study were 
developed to assess the reading and writing achievement levels 
oe scenes The content of the items assessed the reading and 
writing skills as defined by the objectives which were selected 
by the Steering Committee after consultation with the Curriculum 
Branch and Senco in the language arts. Although every 
attempt was saaete sample the language arts programs as defined 
by selected reading and writing objectives, the results of this 
study are limited by the restricted time period designated for 
testing and the number and content of the items which may or may 


not have sufficiently sampled all of the selected objectives. 
2. DESIGN OF THE STUDY 


I.2.1 Introduction | 

rhe design for the study was initially outlined in a study 
proposal. It was faces reviewed and finalized as to details at 
several meetings between CIR personnel and the Steering Committee. 
It should be noted that the design model which was used has pro- 


duced results that describe only the current status of student 


achievement in the province. The results do not show causal rela- 


tionships. 


I.2.2 Sampling Plan 

The Sampling Plan was obtained from the Student aeaiGneion 
and Data Processing Branch of Alberta Education and reviewed by 
CIR personnel and the Steering Committee. Sampled schools were 
initially contacted by Alberta Education and were contacted at a 
later date by CIR. For grades 3 and 6 two stratifying variables 
were used to categorize the schools: 1977-78 enrollment in the 
ose ce reais and type of school system. 

For grades 9 and 12 the decision sae made to geeaciey 
using enrollment only. Schools randomly selected for each enroll- 
ment category were then randomly assigned to various studies, this 
study included. 

The finalized sample for hie Language Arts study was com- 
prised of the following: 102 schools at the grade 3 level, 101 
_ schools at the grade 6 level, 56 schools at the grade 9 level, and 
25 schools at the grade 12 level. 

The 1977-78 enrollment lists of the schools in the province 
were used to determine the total number of students to be tested. 
Based on Gnade enrollment lists, tests were sent to the schools. 
Half of the tests wereereaaing tests and half were writing tests. 
The total usable returns from the schools were as follows: 1,408 


in grade 3 reading, 1,385 in grade 3 writing; 1,597 in grade 6 


reading, 1,598 in grade 6 writing; 1,464 in grade 9 reading, 1,495 
in grade 9 writing; 915 in grade 12 reading and 899 in grade 12 


“writing. 


I.2.3 Student Outcomes/Objectives 

| oe Gaseass of the study it was necessary to obtain a 
list of measurable objectives in reading and writing at grades 3, 
6, 9 and 12 vanpeetiuely: From those lists of objectives available 
from Alberta paudieton by September 30, 1977, it was determined 
that additional oak would have to be done in order to produce 
lists of measurable objectives. It was agreed at a Steering 
Committee Meeting that CIR personnel and consultants would produce 
comprehensive lists of student outcomes/objectives by reviewing 
current eeeieuion guides, skill lists, and authorized textbooks. 
These lists of outcomes/objectives were produced and were ranked 
by the Steering Committee as to the desirability and feasibility 
of their assessment. The provincial Language Arts Coordinating 
Committee also ranked ced objectives using the same criteria. 
Opportunity was given for the inclusion of additional objectives. 

Based on the results of this ranking, finalized lists of 

selected objectives in reading and writing were used as a basis 
for pisanenng. ana selecting assessment items and exercises for 
the testing. The comprehensive lists of student outcomes/objectives 


have been included as an appendix to Part II of this report. 


I.2.4 Instrument Development 

A preliminary step in the development of the objective- 
based instruments was to review existing reading and writing tests 
and to consider the possibility of adapting them for use in this 
Study. It was concluded that none of the available tests, in 
their entirety, assessed the selected reading and eteine objec- 
tives. Therefore, the strategy used to produce the instruments 
was to adapt items where available - to develop new items where 
necessary. All eae were then reviewed by asnsnitente and the 
Steering Committee, and revisions made as indicated. 

For each pilot reading test a total of four multiple-choice 
items were ee ee for each objective. The Sreceh esas section 
of each phan weiss test included four items for each objective. 
Each writing test also included a writing pedaetion task (extended 
discourse). Administrators' Manuals were developed for use with 
each of the instruments. 

The pilot testing was conducted to investigate the admini- 
stration procedures and the items of the proposed tests. The pilot 
testing involved a total of 809 students in grades 3, 6, 9 and 12 in 
a total of 32 classrooms in 16 urban and rural schools in Calgary 
and area which had not been gelacted for inclusion in the MACOSA 
Studies. Arrangements for testing were made with each district 
Superintendent and school principal. A pilot test team administered 
the tests to the students. All ee test answers were recorded 


by the students on computer answer sheets which were later 
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machine-scored by Alberta Education. All writing test answers 

were recorded by the students in the test booklets. These booklets 
were later hand-scored by a team of markers who recorded the results 
on computer sheets. No difficulty in using the testing format was 
experienced by either the students or the testing team. Comments 
from students and teachers concerning the tests were also collected 
by the pilot testing team. An item analysis was computed for the 
reading tests and an error count was computed for the writing test. 
Letters of appreciation were sent by CIR to all schools who partici- 
pated in the pilot testing. 

The Steering Committee and CIR staff made revisions to the 
pilot test instruments and Administrators' Manuals using the results 
of the item analysis, the error count, observations made by the 
pilot testing team, comments from students and teachers, and obser- 
vations made by the markers. Camera-ready copies of the finalized 


instruments were produced and printed. 


I.2.5 Data Collection 

A letter dated February 8, 1978, was sent by the Associate 
Deputy Minister of Education to the school superintendents in 
Alberta. The request was made for the superintendents to ask for 
the cooperation of the sampled schools which were to be included 
in the testing program.: Individual school principals were later 
contacted by CIR and given more specific details concerning the 


testing program. 


Approximately two weeks before the testing date of 
May 17, 1978, tests, computer answer sheets and Administrators' 
Manuals were distributed -to the 284 schools. Distribution was 
made by using the bus lines, couriers, and special delivery mail. 
Sufficient tests were sent to ensure that half the students at 
the specified grade level in each school would write the reading 
test and half the students would write the writing test. Selection 
of students for writing each test was to be done on a random basis. 
Instructions were given that all answer sheets and test booklets, 
used and unused, were to be returned to the Canadian Institute for 


Research. Returned materials were collated at CIR. 


I.2.6 Scoring 

All reading test answer sheets were computer scored by 
Alberta Education. The scoring procedures employed in the marking 
of the writing tests had previously been developed and used to mark 
the pilot writing tests. Minor revisions were made to the eemednge 


keys before the final tests were marked. A team of 20 markers was 


trained to mark the final tests. The proofreading section, which includ- 


ed capitalization, punctuation, spelling, and sentence writing, 
was marked by one marker using a Proofreading Marking Scale for 
each grade. The punctuation and capitalization were scored as 
correct or incorrect; the spelling was scored for identification 
of error as well as correct spelling of identified error; . and 


the sentence writing was scored on scales which identified types 
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of errors in sentence writing and variety in sentence writing. 

Modified holistic scoring of all writing products was 
done by two ene ona sGaleroe five. In the event of a dis- 
crepancy which was greater than one, a third marker scored the 
product. It was determined during the pilot testing that reader- 
agreement for this marking was very high as only three tests out 
of a total of 379 had to have a third aneeg 

seeing of the writing product was also done using a 
Descriptive Marking Scale for each grade. These scales were 


based on the objectives as selected for each grade level. 


I.2.7 Data Analysis Plan 


Basic distributional statistics were computed by Alberta 
Education using the reading and writing test data. Summaries of 
grade level aes eae reading and writing were produced as follows: 
number of students: attempting each item, percentages of students 
correctly answering each item, and percentages of items achieved 


as keyed to objectives. 
3. SUMMARY OF READING RESULTS 


I.3.1. Introduction 
The following chart indicates the organization of the 
reading objective clusters as assessed in the study. The majority 


of the clusters were composed of more than one objective. 
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Chart I-1 


ORGANIZATION OF READING OBJECTIVES* 


Structural 


Meaning 
Word Through 
Level Analysis Contextual 


Sight Vocabulary 


Explicit 
Level 


Connected 


Discourse 


/ Relationships 


Inferential 
Level 


Main Idea 


st 


*For complete lists of objectives, 
see Appendix to Part II of the Report. 
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I.3.2 Reading in Grades 3, 6, 9 and 12 

| Table I-l presents a summary of reading across the grades. 
The figures indicate the average percent es for the items used 
to test each cluster of sieeve. Not all clusters of objectives 
were assessed at eh ead level, therefore some columns in the 


table are incomplete. 


Table I-l 


Reading - Grades- 3, 6, 9 and 12 
Male and Female 


Grade Grade Grade 
3 6 9 


Word Level: Meaning Through 
Analysis 


Grade 


I.A.1 Phonetic ; 80.8 


I.A.2 Structural | 78.4 62.3 

I.A.3 Contextual 76.6 70.2 

I.A.4 Sight Vocabulary | 67.6 -- 
Connected Discourse: | 

Explicit Level | 

II.A.1 petaiis | 63.1 == 

II.A.2 Relationships | 65.5 70.0 

TI.A.3 Main Idea | 67.9 76.3 
Connected piseGueaee 

Inferential Level 

II.B.1 Details 67.5 733 

II.B.2 Relationships 77.9 57.4 

II.B.3 Main Idea | 66.7 57.2 74.8 

II.B.4 Outcomes 72.5 

II.B.5 Purpose 71.0 

II.B.6 Tone 
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In the Word Level category, grade 3 students achieved an 
average percent correct of 80.8 when using phonetic analysis. This 


percentage represents the highest achievement score in the reading 


assessment. Structural analysis involved decoding unfamiliar words 
through the use of structural elements. Grade 6 students achieved 


an average of only 47.3 percent of the items. The achievement 


levels in Contextual analysis were Similar at all grade levels. 
Sight Vocabulary recognition at grades 3 aus 6 was 67.6 percent 
and 68.3 percent respectively even though the vocabulary tested 
was isotenpviesres the grade level. 

The Connected Discourse items were based on reading passages. 
The Explicit Level involved the more actual levels of the passage. 
The Inferential ee required a higher level of thinking as the 
student had a make inferences an judgments. 

Several of the averages in the Connected Discourse cate~ 
gories were low. It appears however that there is an improvement 
through grades 6 to.12 in Outcomes which eived waking sae acaee. 


predictions, and conclusions. 
4. SUMMARY OF WRITING RESULTS 


1.4.1 Introduction 
Chart I-2 indicates the organization of the writing 
objective clusters as assessed in the study. The majority of the 


clusters were composed of more than one objective. 


ee 
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Chart I-2 


ORGANIZATION OF WRITING OBJECTIVES* 


Limits content to topic 


- Develops topic 


A Sentences - Structure & Mechanics 


Paragraph organization 


Extended Paragraph structure 
Discourse 


(Writing 
Product) 


Capitals 


REOot reading Punctuation 


Sentences 


*For complete lists of 
objectives see Appendix to 
Part II of the Report. 
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I.4.2 Wtiting in Grades 3, 6, 9 and 12 

A summary of the results of the writing assessment is pre- 
sented in Table I-2. The three marking scales used to assess the 
writing tests are presented in detail in Part II of the main report. 
The Descriptive scoring data as presented in Table I-2 indicates 
the iweceueaae of writing products which were assessed as a three 
ora four. These ‘scores denoted positive aspects of writing as 
keyed to each of the objectives. The Holistic scoring data as 
presented in the following table indicates those writing products 
which were scored as being acceptable. The Proofreading data 
indicates average percentages correct (no érrors) for items used 
to Aeeecd capi taiieatson: Seeing. punctuation, and sentence 
eee - 

Because all objectives or clusters of objectives were not 
tested at each grade level, several of the columns in Table I-2 
_are incomplete. 

Table I-2 suggests that students in each of the grade levels 
do not experience difficulty in limiting content to the topic when 
producing a writing product. Grade 12, however, had difficulty 
in developing a topic.. 

All percentages were low for Sentences hier assessed the 
ability to write sentences in terms of structure and meshadies; 
Percentages for paragraph organization and structure were also 
low for those grades assessed. 


Except for Grade 12, percentages for vocabulary usage and 
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Table I-2 


Writing - Grades 3, 6, 9 and 12 
Male and Female 


Descriptive Scoring 


I.l Limits content to 
topic 


I.2 Develops topic 


I.3 Sentences (structure & 
mechanics) 


I.4 Paragraph organization 


I.5 Paragraph structure 
I.6 Vocabulary usage 
I.7 Vocabulary breadth 
I.8 . Spelling 

I.9 Composition impact 


I.10 Handwriting 
Holistic Scoring 


Proofreading Scoring 
IIi.l Capitals 
II.2 Spelling 
II.3 Punctuation 


II.4 Sentence Writing 


_ Grade | Grade ~ Grade 
3 6 9 


94.3 93.2 ! 95.0 
79.5 | 84.5 | 71.8 
51.0 39.7 | 61.0 
ame a : 55.7 
ee BO ed : 48.8 
84.6 88.4 71.1 
84.2 84.4 | 86.0 
68.4 69.8 | 83.6 
73.0 63.2 | 68.8 
87.9 | 74.1 | 88.7 
24.7 27.1 | 51.9 
70.4 -- — 
46.5 43.5 55.1 
$69 50.0 35.9 
65.8 27.7 24.2 


Grade 
12 
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Beesaed were consistently higher than several of the other objectives 
assessed. Spelling as assessed in the writing products was lower 
in grades 3 and.6 than in grades 9 and Diese 

Fe eae ae ances eien Impact varied somewhat 
evi quade to grade. HanawElCiAe does not goes to ba a sérious 
problem in the students’ writing.. | | 

In the ab Biscie ecenia, which identified those writing 
products assessed as acceptable in Sie ae tne writing ere 
grade 9 performance was approximately twice that of the other grades. 

In phe euesivaaaine section which paces the mechanics 
used in writing, all scores were low. The Sentence Writing 
evaluated variety in sentence writing as well as seneddes struc- 
ture. The scores seep pategouy correspond savtie sdeuss in 
the Descriptive scoring category where sentence writing in the 
writing product was evaluated for Seviebige and mechanics. 

In summary, it appears that the majority of students are 
in need si saaieional sneeeaeeion in those skills which are necessary 
for the production of an sicapeebie seins produet. Difficulty 
in the mechanics of writing and sentence. writing was indicated in 
the Proofreading section of the assessment. These findings corre- 
spond to findings in the Descriptive and Holistic scoring which 
indicated difficulty in mechanics and sentence writing as well as 
difficulty in other skills such as paragraph organization. 
Vocabulary usage and breadth did not appear to be a problem except 


at the grade 12 level. Students were also able to limit content 


Ee ee nme eT Re eg ene 
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to topic and to develop a topic except at the grade 12 level. 
It appears, in general, that sae students have the 

necessary ideas and vocabulary which are used in the production 

of a writing product, but lack the necessary organizational and 


mechanical skills used in writing. 
5. VARIABLE OF SEX 


I.5.1 Introduction 
Significant differences were computed for the male and 


female performance in the reading and writing clusters. 


I.5.2 Reading Results 

Few significant differences appear to exist between the 
reading performance of males and females. At the grade 3 level 
the results suggest that female performance is higher in struc- 
tural analysis and inferring the main idea, and that male per- 
formance is higher in sight vocabulary. 

At the grade 6 level females excelled in inferring rela- 
tionships and males excelled in structural analysis. 

The performance for three clusters of objectives was 
significantly different in favor of the females at the grade 9 
level: identifying main idea, and inferring outcomes and tone. 

At the grade 12 level females performed higher than males 
in structural analysis: and identifying relationships. 


There does not appear to be any consistent significant 


18 
differences on reading performance between the grade levels. 


I.5.3 Writing Results 

Considerable significant differences in writing performance 
existed in favor of the females at each of the grade levels assessed. 
Except for limiting content to the topic at the grades 3 and 6 
levels, and sentence aie en naenanees at the grade 12 level, 
it appears that in the Descriptive scoring (See Table I-2 ) the 
females performed higher than- the males on the writing product. 
The results of the Holistic scoring of the writing product supported 
the findings in the Descriptive scoring as the females consistently 
performed better than males. 

eeseral: giguaeieane ai erercites. which exist in the Proof- 
reading results are also in sgGe GE the Pemat ees at the grades 
3 eer significant ait tepeaces esta iy seer ang; paneeuas 
tion, and aes eieaie. eiedine gyaae a Teyed Bignieecaae 
differences were evident in spelling ave sentends writing; and at 
the spans 12 ievel eignteican’ qizesicnees existed in spelling 


and punctuation. 


6. RECOMMENDATIONS 
The Contractor recommends that: 
6.1 Assessment of the language arts in the Province of Alberta 
should be an on-going process using the present data as baseline 
data for. comparisons in future assessments. Future assessments 


would include the study of such problems as reliability of the 
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reliability of the tests over time (test-retest studies), relation- 
ship between reading and writing and other studies relating teacher 
activities to pupil growth. There appears to be no reason why all 
pupils in all schools should be tested each time. A stratified 
random sample of students would make an appropriate selection for 


on-going tests. 


6.2 A needs assessment study should be conducted in the 
Province of Alberta to determine terms of reference for language 
arts programs. Such a needs assessment program would provide 
information about what is currently in place in language arts pro- 
grams as well as indicating gaps in such programs. Tests could 
then be matched to both eatin program components and to those 
which need to be put in place. This would address firmly the 


question of the validity of the testing program. 


6.3 Indicators should be established for all the goals of the 
provincial language arts programs so that each facet of language 
growth can be monitored (i.e., cognitive and affective goals). 
This recommendation recognizes that many of the established goals 
of the language arts programs esinee be readily monitored. The 
establishing of indicators for these goals could lead to the 
establishing of monitoring procedures for the indicators. This 


would also have the added advantage of monitoring growth over time. 


6.4 Existing data should be scrutinized to determine whether 


it is feasible to conduct additional research activities using 
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present data. This recommendation suggests that considerable data 
has already been gathered in reading and writing and that additional 


research activities should be designed to capitalize on this data. 


6.5 An item bank should be established for use in ongoing 
wyatadeiod of Gi language arts. Such an item bank would make it 
possible for individual school systems to assess specific areas of 
competence which might be idiosyncratic to their own programs. 
This system is especially useful if flexibility in instructional 


programs is to be maintained. 


6.6 Training sessions should be held for teachers who plan to 
inet and aaaies es tests, because the grading, particularly 
of written materials, is important for the validity of comparisons 
that may be made. This procedure is especially important if the 


tests are to be used for comparisons across schools or over time. 


6.7 Data from test administration should be accumulated from 
year to year. Correlational analyses, relating the results of 
reading and writing to other competency areas, should be conducted 
and presented in the form of expectancy tables. In this way the 


institution will have both criterion and prediction data. 


6.8 While the tests were designed for grades 3, 6, 9 and 12, 
it would be appropriate to try them at surrounding grade levels. 
Since items and goals differ within the present tests, this pro- 


cedure would allow for the analysis of developmental trends. 
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6.9 Additional demographic data should be included in future 
assessments. This recommendation refers to the fact that little 
demographic data was collected in the present study. Additional 
variables such as geographic location, attitudes, value of language 


arts, etc. could be useful in future assessments. 


6.10 The test administration procedures with minor revisions, 
should be repeated in the next assessment. This recommendation 
arises from the observations made by the Contractor in regard to 


test administration procedures. 


